On Matching Forensic Sketches to Mugshot Photos Under review TPAMI

The problem of matching a forensic sketch to a gallery of mugshot images is addressed in this paper. Previous research in sketch matching offered solutions to matching highly accurate sketches that were drawn while looking at the subject. We refer to these as viewed sketches. Forensic sketches differ from viewed sketches in that they are drawn by a police sketch artist using the description provided by an eye-witness who is typically unfamiliar with the subject. To solve the problem of matching forensic sketches, we propose a robust framework, called local feature-based discriminant analysis (LFDA). In LFDA, we first represent both sketches and photos using gradient location and orientation histograms and multi-scale local binary patterns. A discriminant projection method is then used on the feature-based representation for minimum distance matching. We apply the LFDA method to match a dataset of forensic sketches against a mugshot gallery containing 10,159 images. Compared to a leading commercial face recognition system, LFDA offers substantial improvements to matching forensic sketches to the corresponding images. We are able to further improve the matching performance using race and gender information to reduce the target gallery size. Experiments on matching viewed sketches also demonstrate that LFDA matching method exceeds all other previous methods of sketch matching.